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Why GAO Did This Study 

The Transportation Security 
Administration (TSA) is 
responsible for screening all 
checked baggage in U.S. airports 
for explosives and has deployed 
explosive detection systems and 
developed standard procedures for 
their use. TSA also allows 
alternative screening procedures to 
be used for short-term, special 
circumstances. This report 
addresses (1) how TSA prioritized 
the use of checked baggage 
screening procedures and 
identified trade-offs in security 
effectiveness and operational 
efficiencies; (2) how TSA reported 
use of the procedures and ensured 
that standard procedures are used 
whenever possible; and (3) what 
steps TSA took to reduce airports' 
need to use alternative screening 
procedures and to establish 
performance measures to monitor 
their use. To address these issues, 
GAO interviewed TSA officials, 
reviewed information from TSA's 
database on checked baggage 
screening operations; and 
conducted airport site visits. 
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GAO is recommending that TSA 

use information on airport usage of 

alternative screening procedures in 

conducting covert testing; 

strengthen TSA's monitoring and 

tracking of the use of alternative 

screening procedures; and develop 

performance measures and targets 

for the use of alternative screening 

procedures. DHS reviewed a draft 

of this report and generally 

concurred with GAO's findings and 

recommendations. 
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To view the full product, including the scope 
and methodology, click on the link above. 
For more information, contact Cathleen 
Berrick at (202) 512-3404 or 
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What GAO Found 

TSA has prioritized standard and alternative checked baggage screening 
procedures based on legislative requirements and TSA officials' judgment of 
the procedures' effectiveness. Use of various procedures to screen checked 
baggage has involved trade-offs in security effectiveness, which vary by the 
type of procedure used and the circumstances of its use. It is TSA's policy to 
use standard procedures whenever possible because TSA officials 
determined that these procedures provide the most effective detection of 
explosives. TSA policy also allows the use of alternative screening 
procedures when volumes of baggage awaiting screening pose security 
vulnerabilities or when TSA airport officials determine that there is a 
security risk associated with large concentrations of passengers in an area 
waiting for their baggage to be screened. Regarding operational efficiencies, 
TSA has not fully determined the throughput and costs of the various 
alternative screening procedures in part because it does not count the 
number of bags screened using these procedures. TSA has conducted covert 
tests (undercover, unannounced) of standard procedures, but has not 
conducted this testing for alternative screening procedures. TSA cited 
logistical difficulties in conducting covert tests for alternative screening 
procedures. However, by not doing so, TSA is not collecting data that could 
provide useful information in determining the security effectiveness of the 
procedures in an operational setting and how to improve their effectiveness. 

TSA cannot identify the percentage of checked baggage screened using 
standard versus alternative screening procedures because TSA records 
standard procedures in terms of the number of bags screened in its 
management information system, but records alternative procedures in 
terms of the number of occasions and hours of use. However, TSA officials 
estimated that a low percentage of checked baggage is screened using 
alternative screening procedures. To assess the extent that standard 
screening procedures are used whenever possible, TSA has established 
internal controls to monitor the use of standard and alternative screening 
procedures; however, these controls were not always implemented to ensure 
the gathering of complete and accurate information. This may limit TSA 
managers' ability to assess the effect of using alternative screening 
procedures and determine what should be done to minimize the use of the 
procedures. TSA headquarters officials stated that they are working with 
TSA airport staff to correct such reporting problems. 

TSA has taken steps to reduce the need to use alternative screening 
procedures at airports, including anticipating factors that could increase 
passenger and baggage volume and acting to address these factors. 
However, TSA has not developed performance measures and targets to 
assess its progress in minimizing the need to use the procedures. By creating 
performance measures, TSA could gauge whether it is making progress 
toward minimizing the need to use alternative screening procedures at 
airports. Performance targets for the procedures would be an indicator of 
how much risk TSA is willing to accept in using the procedures. 
United States Government Accountability Office 
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^ G A O 

— Accountability * Integrity * Reliability 

United States Government Accountability Office 
Washington, DC 20548 



July 28, 2006 

The Honorable James L. Oberstar 

Ranking Democratic Member 

Committee on Transportation and Infrastructure 

House of Representatives 

Dear Mr. Oberstar: 

Commercial U.S. aircraft have long been a target for terrorist attacks 
through the use of explosives carried in checked baggage, as 
demonstrated by the 1988 bombing of a U.S. aircraft over Lockerbie, 
Scotland. After the terrorist attacks of September 11, 2001, which further 
highlighted the vulnerability of U.S. aircraft to acts of terrorism, Congress 
passed and the President signed into law the Aviation and Transportation 
Security Act (ATSA), mandating, among other things, that all checked 
baggage at U.S. airports be screened using explosive detection systems by 
December 31, 2002. 1 In response to this mandate, the Transportation 
Security Administration (TSA) has deployed two types of systems and has 
established standard procedures for their use: (1) explosive detection 
systems (EDS) that use specialized X-rays to detect characteristics of 
explosives that may be contained in baggage as it moves along a conveyor 
belt and (2) explosive trace detection (ETD) systems, whereby an 
individual (i.e., a baggage screener) swabs a bag and then inserts the swab 
into the ETD machine, which, in turn, can detect chemical residues that 
may indicate the presence of explosives within a bag. 2 TSA also allows 
alternative screening procedures to be used when volumes of baggage 
awaiting screening pose security vulnerabilities or when TSA officials 
determine that there is a security risk associated with large concentrations 
of passengers in an area. These alternative screening procedures include 
the use of EDS and ETD machines in nonstandard ways, 3 and also include 
three procedures that do not use EDS or ETD — screening with explosives 
detection canines, physical bag searches, and matching baggage to 



Congress subsequently extended this deadline by 1 year. 

2 TSA interpreted ATSA's reference to "explosive detection systems" to allow for the 
deployment of EDS and ETD to satisfy the mandate. 

3 The nonstandard ways that the machines are used is sensitive security information. 
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passenger manifests to confirm that the passenger and his or her baggage 
are on the same plane. It is TSA's policy to use standard EDS and ETD 
screening procedures whenever possible and, when necessary, to use EDS 
or ETD-based alternative screening procedures before using non-EDS or 
ETD alternative screening procedures. TSA has also established operating 
procedures for checked baggage screening that instruct Transportation 
Security Officers (TSO — formerly known as screeners) in how to use these 
standard and alternative screening procedures. 

In February 2004, we testified that TSA was unable to fully utilize 
explosive detection systems to satisfy the ATSA mandate to screen 
100 percent of checked baggage for explosives because of TSO shortages 
and problems with screening equipment, among other factors. 4 Further, in 
February 2005, we reported that TSA considers one baggage screening 
method — the use of EDS machines — to be the superior baggage screening 
procedure in terms of efficiency compared to ETD because EDS 
automatically detects explosives without direct human involvement and 
screens more bags for explosives per hour. We also reported that at most 
smaller airports, where EDS machines are not installed, TSA screens 
solely with ETD machines. Finally, we reported that while TSA had made 
progress in deploying EDS and ETD machines, it had not conducted a 
systematic, prospective analysis of the optimal deployment of these 
machines to achieve long-term savings and enhanced efficiencies and 
security. In February 2006, TSA issued a report to Congress detailing its 
strategic planning framework for its checked baggage screening program. 
According to TSA, the framework, which focuses on identifying optimal 
checked baggage screening solutions for airports, will be used to establish 
a comprehensive strategic plan for TSA's checked baggage screening 
program. TSA expects to complete the strategic plan in early fall 2006. 

You asked that we continue to assess TSA's progress in enhancing the 
effectiveness of checked baggage screening operations. In February 2006, 
we issued a report that contained sensitive security information regarding 
TSA's use of standard and alternative checked baggage screening 
procedures, including the extent to which the procedures were used by 
TSA and the trade-offs in security effectiveness of using standard and 
alternative screening procedures to screen checked baggage for 



4 GAO, Aviation Security: Challenges Exist in Stabilizing and Enhancing Passenger and 
Baggage Screening Operations, GAO-04-440T (Washington, D.C.: Feb. 12, 2004). 



Page 2 



GAO-06-869 Aviation Security 



explosives. 5 We concurrently issued a classified correspondence on our 
analysis of the results of TSA's checked baggage screening covert testing. 6 
This report provides the results of our February 2006 report with sensitive 
security information removed. In this report, we address the following 
questions: (1) How did TSA prioritize the use of standard and alternative 
checked baggage screening procedures and what security effectiveness 
trade-offs and operational efficiencies has TSA identified in using these 
procedures to screen checked baggage for explosives? (2) To what extent 
has TSA used standard and alternative screening procedures to screen 
checked baggage for explosives and how does TSA ensure that standard 
screening procedures are used whenever possible? (3) What steps has TSA 
taken to reduce airports' need to use alternative screening procedures and 
to establish performance measures and targets for the use of the 
procedures? 

To address how TSA prioritized the use of checked baggage screening 
procedures and identified trade-offs in security effectiveness and 
operational efficiencies of using standard and alternative screening 
procedures to screen checked baggage for explosives, we assessed TSA's 
standard operating procedures, obtained and analyzed relevant legislation, 
and conducted a literature search to obtain information on screening 
procedures, technologies, and related aviation trends. We also reviewed 
studies conducted by TSA's Transportation Security Laboratory regarding 
checked baggage screening. We interviewed officials from various TSA 
offices as well as air carriers, explosive detection systems equipment 
manufacturers, and an airport industry association to obtain information 
regarding TSA's checked baggage screening procedures and the relative 
priority they were given. To determine the extent to which TSA used 
standard and alternative screening procedures to screen checked baggage 
and how TSA ensured that standard screening procedures are used 
whenever possible, we reviewed TSA's checked baggage standard 
operating procedures manual and its Performance Management 
Information System (PMIS) database, which contains information on 
checked baggage screening operations. 7 We compared TSA procedures for 



b GAO, Aviation Security: TSA Management of Checked Baggage Screening Procedures 
Could Be Improved, GAO-06-291SU (Washington, D.C.: Feb. 28, 2006). 

6 GAO, Analysis of TSA's Covert Testing for Checked Baggage Screening, GAO-06-317C 
(Washington, D.C.: Feb. 28, 2006). 

7 PMIS is a Web-based application used by TSA field staff to submit data to headquarters on 
operations and performance. 
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use of standard and alternative screening procedures with the Comptroller 
General's Standards for Internal Control in the Federal Government. We 
interviewed TSA officials to determine the reliability of the information 
contained in the PMIS database and reviewed selected database records. 
Although we determined that the database did not always contain accurate 
data on the duration of each occurrence of alternative baggage screening 
procedures, we nonetheless considered the database to be sufficiently 
reliable for purposes of this report since the data identified overall trends 
in the use of alternative screening procedures. To assess the steps TSA has 
taken to reduce airports' need to use alternative screening procedures and 
the measures and targets TSA has set for alternative screening procedures, 
we analyzed TSA directives regarding use of the procedures, PMIS data, 
standard operating procedures for checked baggage, and the inventory of 
explosive detection systems. When we found discrepancies in the 
inventory data of explosive detection systems, we worked with TSA to 
resolve the discrepancies. TSA also completed a reconciliation of the 
inventory database with data collected manually by TSA officials. We also 
examined TSA's checked baggage screening performance measures and 
targets as they relate to the Government Performance and Results Act of 
1993 (GPRA) requirements. We visited nine airports during this review to 
observe baggage screening operations. We chose these airports based on 
reporting in PMIS, selecting some airports that had used alternative 
screening procedures and some that had not used the procedures. We also 
conducted structured interviews with TSA Federal Security Directors 
(FSD) 8 and their staffs who were responsible for the security of 
29 randomly selected airports. Because we selected a nonprobability 
sample of airports to visit, the information we obtained during these visits 
cannot be generalized to all airports nationwide. 9 Also, while the 
interviews we conducted with FSDs were random samples of airports, the 
samples were too small to generalize the interview results with a high 
degree of statistical confidence to all airports nationwide. 



8 The Federal Security Director is the ranking TSA authority responsible for the leadership 
and coordination of TSA security activities at the nation's commercial airports. 

9 Nonprobability sampling is a method of sampling where observations are selected in a 
manner that is not completely random, usually using specific characteristics of the 
population as criteria. Results from nonprobability samples cannot be used to make 
inferences about a population because in a nonprobability sample, some elements of the 
population being studied have no chance or an unknown chance of being selected as part 
of the sample. 
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We conducted our work from August 2004 through July 2006 in 
accordance with generally accepted government auditing standards. A 
more detailed discussion of our scope and methodology is contained in 
appendix I. 



RGSllltS in Brief prioritized the use of standard and alternative checked baggage 

v >v * ' " " screening procedures based on legislative requirements and TSA officials' 

judgment of the effectiveness of the procedures, and TSA's use of these 
procedures has involved trade-offs in security effectiveness. It is TSA's 
policy to use standard EDS and ETD screening procedures whenever 
possible because of legislative requirements to do so and because TSA has 
concluded that these procedures provide the most effective detection of 
explosives at a checked baggage screening station. TSA considers 
screening with EDS to be superior to screening with ETD because EDS 
machines process more bags per hour and automatically detect explosives 
without direct human involvement. Given the high volumes of checked 
baggage processed at some airports and unforeseen events such as 
equipment failures and severe weather, TSA has determined that it will 
have to continue to at times use alternative screening procedures to 
screen checked baggage for explosives. TSA officials prioritized the use of 
these procedures based on their professional judgment of the 
effectiveness of the procedures and the classified probabilities of 
detection of EDS and ETD machines obtained from the TSA 
Transportation Security Laboratory. Alternative screening procedures 
include physical bag searches; alternative hybrid procedures, which 
involve a combination of the standard checked baggage procedures for 
EDS and ETD; 10 matching checked bags to passenger manifests to deter 
bombers who try to load a bag on a plane without boarding; explosive 
detection canines; and another screening method TSA has determined to 
be sensitive security information. With regard to operational efficiencies, 
TSA has not determined the throughput (number of checked bags 
screened per hour) and costs of the various alternative screening 
procedures in part because it does not count the number of bags screened 
using these procedures. While TSA has used alternative screening 
procedures for more than 3 years and expects to continue to use them, it 
has not tested the security effectiveness of these procedures in an 



Alternative hybrid procedures use a combination of EDS and ETD at a screening station 
that is usually configured only for EDS. At some screening stations, TSA has also allowed 
the use of a hybrid configuration as a standard screening procedure. 
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operational environment. TSA has conducted national covert (undercover, 
unannounced) testing of standard screening procedures since September 
2002 to assess checked baggage screening technologies and procedures, 
and TSO performance in detecting explosives in an operational 
environment. TSA has also authorized FSDs and their designates to 
conduct local covert testing on standard screening procedures at airports 
since March 2005 to determine if TSOs can detect simulated improvised 
explosive devices hidden in checked baggage. However, TSA has not 
conducted national or local covert testing specifically focused on 
alternative screening procedures. TSA officials stated that conducting 
national testing of alternative screening procedures would be challenging, 
particularly since testing is planned ahead of time and it is difficult to 
predict when an airport might use alternative screening procedures. While 
it may be logistically difficult to conduct national covert testing on 
alternative screening procedures, TSA data on which airports most 
frequently use the procedures and the reasons for the usage could provide 
TSA with information to use in selecting airports for covert testing as part 
of the risk-based approach to covert testing that it is developing. TSA 
officials also cited challenges in conducting local covert testing of 
alternative screening procedures, including the difficulty in predicting the 
need to use the procedures and the lack of available federal staff to 
conduct the testing, particularly at smaller airports. Because FSDs and 
their staffs authorize and initiate the use of alternative screening 
procedures, they could schedule some of their ongoing local covert testing 
for checked baggage screening to coincide with the use of these 
procedures. In not assessing the alternative screening procedures through 
covert testing, TSA is not collecting data that could help determine how 
effective the procedures are in an operational setting or how to improve 
the security effectiveness of the procedures. 

TSA reported using both standard and alternative screening procedures in 
PMIS but could not identify the percentage of all checked baggage 
screened using EDS and ETD with standard or alternative screening 
procedures. TSA could not identify the percentage of usage of standard or 
alternative screening procedures because for standard screening 
procedures, PMIS contains data on the number of bags screened; whereas 
for alternative screening procedures, it contains data on the number of 
occasions and hours of use. However, TSA officials estimated that a high 
percentage of checked baggage is screened using EDS and ETD machines 
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with standard screening procedures 11 and a low percentage is screened 
using alternative screening procedures. 12 TSA also established internal 
controls to monitor and track the use of standard and alternative 
screening procedures, including requiring FSDs to report the use of 
alternative screening procedures into PMIS and to call headquarters for 
permission to use the alternative screening procedures for more than 
2 hours. According to the Comptroller General's Standards for Internal 
Control, all transactions and other significant events need to be 
completely and accurately documented. However, we identified several 
areas where TSA's information on its use of standard and alternative 
screening procedures was not complete or accurate for the period covered 
by our review. First, TSA's method for estimating the number of bags 
screened with ETD using standard screening procedures led to inaccurate 
counts of baggage screened. Improved counting of bags would provide 
TSA management with better information to use in making decisions 
related to its baggage screening operations including where to deploy 
screening equipment. Second, FSDs and their staffs did not always 
accurately report the occurrences when a particular alternative baggage 
screening procedure was used, impeding TSA's ability to reliably 
determine how often and for how long the alternative screening 
procedures were used. Third, FSDs and their staffs did not always report 
the use of alternative screening procedures as required. Inaccurate and 
incomplete reporting on how often alternative screening procedures are 
used or for how long, combined with not tracking the number of bags 
screened using the procedures, may limit TSA managers' ability to assess 
the effect of using alternative screening procedures on aviation security 
and to determine the types of actions that should be taken at airports to 
help minimize the use of these procedures. TSA officials stated they are 
working with FSDs to correct these reporting problems and have issued 
guidance clarifying requirements for reporting alternative screening 
procedures. 



TSA does not require airports to report information on the number of bags screened using 
alternative screening procedures. To derive its estimate of use of alternative screening 
procedures across the system, TSA used alternative screening procedures baggage counts 
only from those airports that voluntarily reported the information in order to calculate the 
average number of bags screened per hour. TSA then used this calculation of the average 
bags per hour and the total number of screening hours using the procedures to make this 
estimate. Because TSA did not have baggage counts for all of the occurrences of alternative 
screening procedures, this estimate may be inaccurate. 

12 TSA determined that the estimated use of alternative screening procedures is sensitive 
security information. 
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TSA has taken steps to reduce airports' need to use alternative screening 
procedures, but has not established performance measures or targets 
regarding the use of these procedures. According to our review of PMIS 
data, the use of alternative screening procedures between October 2004 
and September 2005 — measured in terms of the total hours of use reported 
by FSDs — initially increased and then declined. 13 TSA attributed the 
reported overall decline in the use of alternative screening procedures to a 
number of factors, including better coordination with groups such as tour 
operators; deploying "optimization teams" to airports that were frequently 
using alternative screening procedures to determine why the procedures 
were being used so often and to suggest remedies; and deploying 
additional EDS machines. 14 As we reported in March 2005, additional EDS 
systems integrated into the airport baggage handling system ("in-line" EDS 
systems) could reduce by 78 percent the number of baggage TSOs and 
supervisors needed to screen checked baggage at airports with the 
systems. 15 TSA's February 2006 checked baggage screening planning 
framework includes a prioritization of which additional airports should 
receive new funding for in-line systems and a description of how stand- 
alone EDS machines from those airports should be redistributed to other 
airports. After in-line EDS systems are installed and any staffing 
reductions are achieved, redistributing the screening positions to other 
airports with staffing shortages could also reduce the need to use 
alternative screening procedures at these airports. While TSA data indicate 
that the use of alternative screening procedures is declining, the strategic 
planning framework states that at some airports alternative screening 
procedures will increasingly be used because of rising passenger traffic. 
TSA has projected that the number of originating domestic and 
international passengers will rise by about 127 million passengers over 
current levels by 2010, which could increase airports' need to rely on 
alternative screening procedures in the future in the absence of additional 
or more efficient EDS machines. Furthermore, while TSA has taken steps 
to reduce the need to use alternative screening procedures at airports, it 



The specific details on the number of hours alternative screening procedures were used 
are sensitive security information. 

14 At 46 airports, a combined total of 154 EDS machines were added; at 22 airports, a 
combined total of 62 EDS machines were removed; and 47 airports maintained the same 
number of EDS machines. One airport with two EDS machines in inventory in 2005 was not 
on the 2004 or 2002 inventory lists. 

15 GAO, Aviation Security: Systematic Planning Needed to Optimize the Deployment of 
Checked Baggage Screening Systems, GAO-05-365 (Washington, D.C.: Mar.15, 2005). 
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has not created performance measures or targets regarding its progress in 
minimizing the need to use alternative screening procedures at airports. 
By creating performance measures for the use of alternative screening 
procedures and corresponding targets, TSA could gauge whether it is 
making progress in working to minimize the need to use alternative 
screening procedures at airports and to consider the necessity to take 
further steps to minimize the need for their use. 

To help strengthen TSA's management of checked baggage screening 
operations, including screening with alternative screening procedures, we 
are recommending that the Secretary of the Department of Homeland 
Security (DHS) direct the Assistant Secretary, TSA, to use PMIS data on 
the use of alternative screening procedures at airports to help determine 
which airports to conduct national covert testing at and when to conduct 
such testing, to conduct local covert testing of alternative screening 
procedures, to strengthen its monitoring and tracking of the use of 
alternative screening procedures, and to develop performance measures 
and performance targets for the use of alternative screening procedures. 

We provided a draft copy of this report to DHS for review. DHS, in its 
written comments, generally concurred with our findings and 
recommendations and stated that the recommendations and findings will 
help strengthen TSA's management of checked baggage screening 
operations. The full text of DHS's comments is included in appendix II. 



Background 



Standard Procedures for ATSA mandated that the screening of all checked baggage at commercial 
Using EDS and ETD airports be done using explosive detection systems by December 31, 2002. 

To satisfy this mandate, TSA deployed two types of screening equipment 
to all airports in the United States where screening is required: 
(1) explosive detection systems, which use computer-aided tomography 16 
X-rays adapted from the medical field to automatically recognize the 
characteristic signatures of threat explosives, and (2) explosives trace 
detection systems, which use chemical analysis to detect traces of 



Computer-aided tomography is a method of producing a three-dimensional image of the 
internal structures of a solid object by the observation and recording of the differences in 
the effects on the passage of waves of energy impinging on those structures. 
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explosive materials' vapors and residues. As we reported in February 2004, 
largely because of shortages of equipment and insufficient time to modify 
airports to accommodate EDS machines, TSA was unable, at certain 
airports, to meet this deadline. Recognizing the obstacles encountered by 
TSA, the Homeland Security Act of 2002, in effect, subsequently extended 
the deadline for screening all checked baggage for explosives until 
December 31, 2003, for airports at which TSA was unable to meet the 
earlier deadline established by ATSA. We also reported that TSA fell short 
of fully satisfying the extended 2003 mandate and continued to face 
challenges in screening checked baggage because of (1) an insufficient 
number of TSOs to operate the EDS and ETD machines, TSO absenteeism, 
and a lack of TSO training in how to operate the machines and (2) a lack 
of EDS and ETD equipment and inoperable equipment. 17 

By taking the equivalent of hundreds of X-ray pictures of a bag from 
different angles, the EDS machine examines the objects inside of the 
baggage to identify the characteristic signatures of threat explosives such 
as density and atomic number. TSA has certified, acquired, and deployed 
EDS machines manufactured by three companies. EDS machines can be 
installed in airports either in stand-alone mode (not integrated with 
baggage handling systems) or in-line (integrated with baggage handling 
systems). TSA has developed standard procedures for using EDS. Figure 1 
shows EDS machines in use at an airport. 



17 GAO-04-440T. 
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Figure 1: EDS Machines In a Stand-alone Configuration Used by TSA to Screen 
Checked Baggage 




Source: GAO. 



At airports that do not have EDS machines and at airports where certain 
screening stations do not have EDS — for example at curbside check-in 
stations — TSA uses ETD machines to conduct primary screening. ETD 
machines are also used for secondary screening, which resolves alarms 
from EDS machines that indicate the possible presence of explosives 
inside a bag. TSA has certified, acquired, and deployed ETD machines 
from three manufacturers. Figure 2 shows an ETD machine in use at an 
airport. 



Page n 



GAO-06-869 Aviation Security 



Figure 2: ETD Machine Used by TSA to Screen Checked Baggage 




Source: GAO. 



At some screening stations, TSA has also allowed primary screening with 
both EDS and ETD machines simultaneously. In this hybrid configuration, 
the EDS machine is used to maximum capacity before the ETD machines 
are used. Additionally, when the EDS machine alarms, the ETD machines 
are also used for secondary screening. 



Alternative Screening TSA also uses alternative screening procedures to screen checked baggage 

Procedures f° r explosives under certain short-term special circumstances, when the 

standard procedures using EDS and ETD are not used. Two of these 
procedures involve the use of EDS and ETD. The first of these is an 
alternative hybrid procedure that is used at a screening station configured 
only for EDS primary screening. Under this procedure, the EDS is to be 
used to capacity and the remainder of the bags are screened with ETD. 
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The specifics of the second procedure that involves the use of EDS and 
ETD are sensitive security information. 18 If one of these EDS- or ETD- 
based alternative screening procedure is not available, TSA will resort to 
one of the procedures that does not use EDS or ETD — canine screening; 
physical inspections of baggage; and positive passenger bag match, which 
requires that passengers be on the same aircraft as their checked 
baggage. 19 

FSDs and their designates not lower than the Assistant FSD for Screening 
may authorize the use of alternative screening procedures under two 
circumstances: (1) when the FSD or his or her designate determines that 
there is a security threat created by large concentrations of passengers 
waiting to have their baggage screened or (2) volumes of baggage awaiting 
screening in a confined baggage screening area pose an explosive or other 
security vulnerability. These circumstances may arise for reasons such as 
high passenger volumes, screening machine breakdowns, or unusual 
weather events such as hurricanes. After alternative screening procedures 
have been used, TSA requires that information on each occurrence be 
recorded by FSD staff into the PMIS database, including circumstances 
leading to the use of the procedure, type of procedure used, and duration. 
This information on the use of alternative screening procedures from PMIS 
is to be included in daily briefing reports for TSA senior management. 



TSA also moved additional ETD machines to screening stations to allow for ETD 
screening. Prior to March 2005, TSA had categorized this procedure, "additional ETDs," as 
an alternative screening procedure. Beginning in March 2005, TSA began to categorize this 
procedure as a standard screening procedure. We did not include this procedure in our 
analysis of PMIS data on alternative screening procedures. 

19 ATSA, as codified at 49 U.S.C. §44901(d)-(e), authorizes TSA to screen checked baggage 
using canine screening, physical inspection, or a bag match program if explosive detection 
equipment is unavailable. 
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TSA Prioritized 
Screening Procedures 
Based on Legislative 
Requirements and 
Judgment of 
Effectiveness but Has 
Not Tested the 
Security Effectiveness 
of Alternative 
Screening Procedures 
in an Operational 
Environment 



TSA prioritized standard and alternative checked baggage screening 
procedures based on legislative requirements and TSA officials' judgment 
of the security effectiveness of the procedures. TSA's use of these various 
procedures has involved trade-offs in security effectiveness. TSA officials 
determined that in general, standard screening procedures are more 
effective than alternative screening procedures. TSA has estimated that in 
terms of efficiency, EDS processes more bags per hour than ETD. With 
regard to operational efficiencies, TSA has not determined the throughput 
and costs of the various alternative screening procedures, in part because 
it does not count the number of bags screened using the procedures. 
Additionally, while TSA has assessed the security effectiveness of 
screening with standard procedures in an operational environment 
through covert testing, it has not conducted similar testing of alternative 
screening procedures. 



TSA Prioritized Standard 
and Alternative Screening 
Procedures Based on 
Legislative Requirements 
and TSA Officials' 
Judgment of Security 
Effectiveness 



TSA is required by legislation to screen all checked baggage using 
explosive detection systems, and TSA officials concluded that standard 
screening procedures that use EDS or ETD provided the most effective 
detection of explosives at a baggage screening station and that alternative 
screening procedures should be used only for short-term, special 
circumstances. These circumstances include times when security targets 
are created by large volumes of passengers awaiting baggage screening or 
when security vulnerabilities are created by volumes of bags awaiting 
screening. According to TSA, a group of officials from its Chief 
Technologist, Chief Counsel, Aviation Operations, and Operations Policy 
offices met to prioritize the use of standard and alternative screening 
procedures. TSA officials stated that this group did not use formal criteria 
to prioritize the procedures but instead prioritized them based on their 
professional judgment of the effectiveness of the procedures, including the 
classified probabilities of detection of EDS and ETD machines obtained 
from the Transportation Security Laboratory. This prioritization was 
subsequently included in TSA's standard operating procedures for 
checked baggage screening. TSA has determined that details on the 
prioritization of alternative screening procedures constitute sensitive 
security information. 
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Standard Baggage 
Screening Procedures Vary 
in Operational Efficiency 



TSA has estimated that, in terms of efficiency, EDS processes more bags 
per hour than ETD — EDS ranges from a minimum of 80 bags per hour for 
one model of a stand-alone machine up to 500 bags per hour for an in-line 
system, compared to 36 bags per hour by the operator of an ETD 
machine. 20 Whenever EDS machines have been installed at a screening 
station, TSA requires airports to use them as the primary method to screen 
checked baggage. At some screening stations, TSA has also allowed the 
use of a hybrid configuration as a standard screening procedure that 
involves colocated EDS and ETD machines for primary screening, with the 
EDS machine used to maximum capacity before the ETD machines are 
used. When the EDS machine alarms, the ETD machines are also used for 
secondary screening. Table 1 shows the bags per hour screened by EDS 
and ETD machines. 



20 TSA officials stated that two TSOs can use an ETD machine at the same time, raising the 
baggage screened throughput to 72 bags per hour 
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Table 1: Bags per Hour Screened Using Standard Screening Procedures for 
Stand-alone and In-line EDS Machines and ETD Machines 


Type of equipment 


Maximum bags per hour 




EDS machines 


Stand-alone 


In-line 


CTX 2500— stand-alone only 


120 


NA a 


CTX 5500 


180 


250 


CTX 9000— in-line only 


NA 


500 


L3 6000 


140 


500 


CT-80 — stand-alone only 


80 


NA 


ETD machines — stand-alone only 


36 


MA 



Source: TSA. 



°NA: Not applicable. 

At 312 mostly smaller airports and at some airport screening stations such 
as curbside check-in stations, TSA has installed ETD instead of EDS for 
primary screening because of the configuration of screening stations, the 
costs associated with procuring EDS, and the low passenger volume at 
smaller airports. In our March 2005 report, we recommended that TSA 
assess the feasibility, expected benefits, and cost to replace ETD machines 
with stand-alone EDS machines for the primary screening of checked 
baggage at those airports where in-line EDS systems would not be either 
economically justified or justified for other reasons. 21 DHS stated that TSA 
was conducting an analysis of the airports that rely heavily on ETD 
machines as the primary checked baggage screening technology in order 
to identify airports that would benefit from replacing ETD machines with 
stand-alone EDS equipment. 22 In February 2006, in response to GAO's 
recommendation and a legislative requirement to submit a schedule for 
expediting the installation and use of in-line systems and replacement of 
ETD equipment with EDS machines, 23 TSA provided its strategic planning 
framework for its checked baggage screening program to Congress. This 
framework introduces a strategy intended to increase efficiency through 



21 GAO-05-365. 

22 The stand-alone EDS equipment TSA is considering for these airports includes surplus 
machines no longer needed once airports installed in-line EDS machines and a newly 
certified EDS machine appropriate for baggage screening operations that require a lower 
throughput (bags screened per hour). 

^Intelligence Reform and Terrorism Prevention Act of 2004, Pub. L. No. 108-458, § 4019, 
118 Stat. 3638, 3721-22. 
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deploying EDS to as many airports as practicable, lower life-cycle costs for 
the program, minimize impacts to TSA and airport/airline operations, and 
provide a flexible security infrastructure for accommodating growing 
airline traffic and potential new threats. 24 The framework is an initial step 
in addressing the following areas: 

• optimized checked baggage screening solutions — finding the ideal mix of 
higher-performance and lower-cost alternative screening solutions for the 
250 airports with the highest checked baggage volumes and 

• funding prioritization schedule by airport — identifying the top 25 airports 
that should first receive federal funding for projects related to the 
installation of explosive detection systems based on quantitative modeling 
of security, economic, and other factors. 

TSA's strategic plan for the checked baggage screening program, which 
TSA expects to complete by early fall 2006, is to include funding and cost- 
sharing strategies for the installation of in-line baggage screening systems. 



Use of Alternative Baggage 
Screening Procedures 
Involves Trade-offs in 
Security Effectiveness, 
while Trade-offs in 
Operational Efficiencies 
Have Not Been 
Determined 



TSA has determined that the use of alternative screening procedures at 
airports has created trade-offs in security effectiveness, but it has not 
determined the operational efficiencies of these procedures in terms of 
throughput and costs. 25 TSA based its prioritization of the alternative 
screening procedures on its judgment of the procedures' security 
effectiveness and classified probabilities of detection of EDS and ETD 
machines obtained from the TSA Transportation Security Laboratory. TSA 
has not determined the operational efficiencies of the various alternative 
screening procedures in terms of throughput and costs in part because it 
does not count the number of bags screened using the procedures. If the 
higher-prioritized alternative screening procedure is not available at a 
screening station, the FSD may authorize a lower-prioritized procedure. 26 



24 TSA has determined that the details of its analysis of the optimal checked baggage 
screening solutions are sensitive security information. 

25 The specifics of the trade-offs in security effectiveness have been determined to be 
sensitive security information. 

26 0ne alternative screening procedure that involves the use of EDS and ETD is not 
discussed in this report because TSA designated the procedure as sensitive security 
information. 
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Positive Passenger Bag Match Under the positive passenger bag match alternative screening procedure, 

TSA coordinates with airlines to ensure that passengers are on the same 
aircraft as their checked baggage. If a passenger checks a bag but does not 
board the airplane, the bag is removed before departure. This procedure 
was first implemented based on the premise that a terrorist would seek to 
place a bomb on an airplane without sacrificing his or her life by boarding 
the airplane. In light of the suicide terrorist attacks of September 11, this 
premise is now considered flawed. An airline trade association and airline 
officials representing two airlines we interviewed also stated that the 
procedure creates operational inefficiencies for airlines. According to 
these officials, the process of matching bags with passengers can delay 
flights because the flight cannot take off until all baggage is matched to an 
on-board passenger. These officials also stated that implementing the 
procedure increases the workload of airline personnel, who are 
responsible for conducting the procedures at the direction of TSA. 

Canine Screening TSA also uses canine units as an alternative screening procedure. These 

units are composed of trained explosives detection canines and handlers. 
In terms of efficiency, TSA officials reported that it can be difficult to 
mobilize canine units in sufficient time to screen checked baggage when 
alternative screening procedures are needed, especially since the need to 
use the procedures can arise without warning. Officials also reported that 
screening checked baggage using canines requires enough open floor 
space to lay out the baggage as well as a sufficient number of personnel to 
move the bags into position for canine screening. 

Physical Inspection The physical inspection alternative screening procedure requires human 

intervention to detect explosives, weapons, and improvised explosive 
devices and their components, and does not involve use of EDS or ETD 
machines. While TSOs are trained to detect improvised explosive devices 
and their components and to detect signs of tampering, the success of the 
TSOs in finding these items depends on their skill in detecting such items 
through manual searches and their adherence to TSA's standard operating 
procedures for checked baggage regarding physical inspection. Since 
human TSOs are involved, the efficiency of physical inspection in terms of 
baggage throughput rate can vary depending on the contents of the bag 
and how quickly the TSO conducts the search. 

Alternative Hybrid Procedures Alternative hybrid procedures involve using a combination of EDS and 

ETD at a screening station normally configured only for EDS. The 
efficiency of alternative hybrid procedures can vary because each use of 
the procedure can involve a different proportion of EDS and ETD 
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screening, with greater use of EDS leading to more efficient screening in 
terms of number of bags screened per hour. 



TSA Has Not Tested the 
Operational Security 
Effectiveness of 
Alternative Screening 
Procedures through Covert 
Testing 



While TSA has reported using alternative screening procedures for more 
than 3 years, it has not tested the security effectiveness of the procedures 
in detecting explosives in an operational environment. TSA has conducted 
national covert testing of standard screening procedures since September 
2002, and local covert testing of standard screening procedures since 
March 2005. However, it has not specifically focused national or local 
covert testing on alternative screening procedures to determine the 
security effectiveness of the procedures. TSA's Office of Inspections (01 — 
formerly the Office of Internal Affairs and Program Review) conducts 
national covert tests at airports to assess the security effectiveness of 
checked baggage screening technology, procedures, and TSO performance 
in detecting explosives in an operational environment. These tests, in 
which undercover inspectors attempt to pass threat objects through 
passenger screening checkpoints and in checked baggage, are designed to 
identify vulnerabilities in passenger and checked baggage screening 
systems and to identify systematic problems affecting screening in the 
areas of training, procedures, and technology. 27 The schedule for this 
testing called for inspectors to test all category X airports once a year, 
category I and II airports once every 2 years, and category III and IV 
airports at least once every 3 years. 28 In August, 2005, TSA suspended this 
cycle of testing. In April 2006, TSA officials stated that OI was moving to a 
testing schedule to include observations of screening stations and 
concentrated testing for improvised explosive devices at the screening 
checkpoint. The schedule is based on risk-based factors such as current 
intelligence information, high-vulnerability airports, procedural changes, 
training initiatives, and introduction of new technologies. According to 
Office of Inspections officials, during the 3-year testing cycle, inspectors 
tested the procedures being used by TSOs at the time of the test; 
alternative screening procedures were tested only if inspectors 
coincidentally conducted a test at a screening station while one of the 
procedures was in use. Office of Inspections officials stated that they did 



'GAO, Aviation Security: Screener Training and Performance Measurement 
Strengthened, but More Work Remains, GAO-05-457 (Washington, D.C.: May 2, 2005). 

28 TSA classifies the over 400 airports in the United States into one of five categories — X, I, 
II, III, and IV. Generally, category X airports have the largest number of passenger 
boardings and category IV airports have the smallest number. 
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not schedule tests of alternative screening procedures because their 
resources were dedicated to conducting testing on standard screening 
procedures. Furthermore, the officials stated that since covert testing 
visits are planned in advance of the tests and the need to use alternative 
screening procedures is not always known in advance, it would be 
logistically difficult to plan a covert testing visit to coincide with an 
airport's use of alternative screening procedures because of airports' 
intermittent and often short-term use of the procedures. While it may be 
logistically difficult to conduct national covert testing on alternative 
screening procedures, PMIS data on which airports most frequently use 
the procedures and the reasons for the usage could provide the Office of 
Inspections with information to select airports for covert testing as part of 
the risk-based approach to covert testing that it is developing. 

In addition to its national covert testing program, in March 2005, TSA also 
began an airport-based local covert testing program to determine if TSA 
checked baggage TSOs can detect a simulated improvised explosive 
device that is hidden in a test bag. Participation in this program is at the 
discretion of the FSD. FSD staff test screening at EDS stations by placing 
simulant explosives in baggage and surreptitiously running the bags 
through the machines. According to TSA, between March 2005 and 
February 2006, 2,526 local tests of EDS screening were conducted at 
108 airports. When we asked TSA headquarters officials in charge of the 
local covert testing program about the feasibility and usefulness of testing 
the use of alternative screening procedures, they stated that they had not 
previously considered testing the procedures through the local covert 
testing program. These officials also cited challenges in conducting such 
testing, including the difficulty in predicting the need to use the 
procedures and the lack of available federal staff to conduct the testing, 
particularly at smaller airports. Because FSDs and their staffs authorize 
and initiate the use of alternative screening procedures, they could 
schedule some of their ongoing local covert testing for checked baggage 
screening to coincide with the use of these procedures. In not testing the 
alternative screening procedures through national or local covert testing, 
TSA is not collecting data that could provide useful information on how to 
improve the security effectiveness of these procedures in detecting 
explosives. 
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The Full Extent of the 
Usage of Alternative 
Screening Procedures 
Is Not Known, and 
Internal Controls for 
Monitoring the Usage 
of Baggage Screening 
Procedures Could Be 
Improved 

TSA reported using both standard and alternative screening procedures in 
PMIS but could not identify the percentage of all checked baggage 
screening using EDS and ETD with standard or alternative screening 
procedures. TSA cannot identify the percentage of usage of standard or 
alternative screening procedures because for standard screening 
procedures, PMIS contains data on the number of bags screened, whereas 
for alternative screening procedures it contains data on the number of 
occasions and hours of use. 29 However, TSA officials estimated that a high 
percentage of checked baggage is screened using EDS and ETD machines 
with standard screening procedures 30 and a low percentage is screened 
using alternative screening procedures. 31 TSA determined that the number 
of bags screened using EDS and ETD with standard screening procedures 



TSA Does Not Collect 
Consistent Data on the Use 
of Standard and 
Alternative Screening 
Procedures to Enable an 
Accurate Determination of 
the Full Extent of Their 
Use 



Although FSDs and their staffs are not required to report the number of bags screened 
with alternative screening procedures into PMIS, some FSDs and staff from airports that 
reported using alternative screening procedures reported this information in comment 
fields on occurrences of use of alternative screening procedures between October 2004 and 
September 2005. The number of FSDs and the number of occurrences have been 
determined to be sensitive security information. 

30 TSA does not require airports to report information on the number of bags screened using 
alternative screening procedures. To derive an estimate of use of alternative screening 
procedures across the system, TSA used alternative screening procedures baggage counts 
only from those airports that voluntarily reported the information in order to calculate the 
average number of bags screened per hour. TSA then used this calculation of the average 
bags per hour and the total number of screening hours using the procedures to make this 
estimate. Because TSA did not have baggage counts for all of the occurrences of alternative 
screening procedures, this estimate may be inaccurate. 

31 TSA determined that its estimate of the use of alternative screening procedures is 
sensitive security information. 
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between October 2004 and September 2005 as well as data on the use of 
alternative screening procedures reported into PMIS during this same 
period are sensitive security information. 



TSA Established Internal 
Controls to Monitor the 
Usage of Standard and 
Alternative Screening 
Procedures, but Some 
Controls Have Not Been 
Adequately Implemented 



Recording of ETD Baggage 
Screening in PMIS 



TSA established internal controls to monitor and track the usage of 
standard and alternative screening procedures, but has not adequately 
implemented some of these controls. An internal control is an integral 
component of an organization's management and is designed to provide 
reasonable assurance that agencies achieve effectiveness and efficiency of 
operations and compliance with applicable laws and regulations. The 
Comptroller General's Standards for Internal Controls require that 
transactions and events be completely and accurately recorded in order to 
ensure that information is available for management to guide operations 
and make decisions. 32 While TSA has established internal controls to 
monitor and track use of baggage screening procedures, such as requiring 
FSDs and their designates to report the use of alternative screening 
procedures into PMIS and to call TSA headquarters for permission to use 
the procedures for more than 2 hours, some of the controls have not been 
adequately implemented because events have not been completely and 
accurately recorded. Consequently, TSA does not have complete 
information on the extent of the use of alternative screening procedures 
that would be helpful for TSA management in making decisions on actions 
to minimize the need to use alternative screening procedures at airports, 
such as deploying screening equipment. 

Information that FSDs and their staffs report in PMIS regarding the 
number of bags screened using ETD machines may not be accurate 
because of the way in which the number of bags screened is estimated. 
While EDS machines automatically count each bag screened, ETD 
machines count each swab analyzed, rather than each bag screened. TSA 
uses this count of analyses to estimate the number of bags screened using 
ETD. The number of analyses for a bag screened using ETD may vary 
depending on how many times the machine alarms during the screening 
process and other factors, which may lead to overreporting of baggage 
screened. TSA officials stated that they are aware of these discrepancies 
and are working to improve counting of baggage screened using ETD. 33 



"GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-2 1.3.1 
(Washington, D.C.: November 1999). 

33 TSA determined that the other factors that may influence the number of analyses 
completed for a bag screened using ETD are sensitive security information. 
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Improved counting of bags would provide TSA management with better 
information to use in making decisions related to its baggage screening 
operations, including where to deploy screening equipment. 

Recording Occurrences of FSDs and their staffs did not always completely and accurately record 

Alternative Screening information in PMIS on the use of various alternative screening 

Procedures procedures. On the basis of our review of PMIS data from October 2004 

through September 2005, we found that FSDs and their designates did not 
always accurately report the occurrences when a particular baggage 
screening procedure was used. For example, some of the airports that 
reported using alternative screening procedures voluntarily reported in a 
PMIS comments field that they used the procedures intermittently over the 
course of several hours, even though in PMIS they reported only one 
occurrence that lasted several hours. 34 FSD staff at one of these airports 
reported in PMIS one occurrence of using alternative screening 
procedures for 15.5 hours straight but reported in the comments field that 
the procedures were used during 24 different occurrences during the 
15.5 hours. According to TSA guidance, these data should have been 
recorded as 24 separate occurrences in the PMIS database, not simply 
noted in the comments field. TSA officials stated that they were aware that 
many airports were reporting the use of alternative screening procedures 
for extended periods of time rather than recording each time the use of the 
procedures was started and stopped within the reported time. In May 2005, 
the TSA Assistant Administrator for Aviation Programs sent a memo to 
FSDs noting that the start and stop time of each individual use of an 
alternative screening procedure at each screening station should be 
reported into PMIS. In our analysis of PMIS data from May 2005 through 
September 2005, subsequent to the issuance of this memo, some of the 
airports continued to report intermittent use of alternative screening 
procedures in the comments fields. 

The design of PMIS also contributed to incomplete and inaccurate 
recording of information because it does not allow FSDs and their 
designates to report two or more alternative screening procedures used 
during the same occurrence. TSA officials have instructed FSDs and their 
designates to record the alternative screening procedure that is used the 
most during the occurrence. One airport voluntarily reported in the PMIS 



34 The comments field is a PMIS database field used to add descriptive data on the 
alternative screening procedures occurrence. TSA determined that the exact number of 
airports that voluntarily reported intermittent use of alternative screening procedures in 
the comments field is sensitive security information. 
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comments field that it used three alternative screening procedures 
throughout the occurrence. However, the PMIS database only allowed 
reporting of the use of one of the procedures during this time. Because of 
these reporting limitations, TSA managers do not receive complete 
information on how often or for how long the various alternative screening 
procedures are actually used — information that could affect their 
decisions on what actions to take to minimize the need to use alternative 
screening procedures at airports. 

Another factor that could contribute to incomplete and inaccurate 
reporting of alternative screening procedures in PMIS is that although 
FSDs and their staffs are required to report every occurrence of the 
procedures in PMIS, they may not have always done so. Until August 2005, 
when this requirement was eliminated, FSDs and their staffs were required 
to report to the Transportation Security Operations Center — TSA's 
command, control, communications and intelligence center — whenever 
they were about to begin using alternative screening procedures or to 
switch back to standard EDS or ETD screening after using alternative 
screening procedures. 35 While TSA officials stated that they did not keep 
formal records of the calls, they kept what TSA termed "informal notes" on 
sheets that included times when the use of the alternative screening 
procedures began and ended and the type of procedure used. When we 
compared a select number 36 of these sheets completed between February 
and March 2005 to PMIS reporting for the same period, we found that 
21 percent of the occurrences of use of alternative screening procedures 
recorded on the sheets were not recorded into PMIS as required by TSA's 
standard operating procedures. 37 Inaccurate reporting on the frequency of 
use of alternative screening procedures may hinder management decision 
making on how best to minimize airport need to use these procedures. 



Permission to Use Alternative While TSA's standard operating procedures require FSDs or their 
Screening Procedures for More designates to call headquarters for permission to use alternative screening 
than 2 Hours procedures that are used for more than 2 hours in order to ensure that the 



°TSA officials stated that they eliminated this requirement because headquarters could get 
information on use of alternative screening procedures through PMIS. 

36 TSA classified the number of sheets that we reviewed as sensitive security information. 

37 We selected sheets to analyze based on the completeness of the information contained on 
the sheets. Each sheet selected had a date, an airport code or name, a beginning and end 
time of use of alternative screening procedures, type of alternative screening procedure 
used, and reason for using the alternative screening procedure. 
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procedures are used only for short-term, special circumstances, TSA does 
not require headquarters to maintain a record of these calls. Because these 
calls are not recorded, TSA management is not able to ensure that the 
requirement in the standard operating procedures is being followed. 
Recording these calls would enable TSA to compare the records to the 
hours of use of alternative screening procedures data maintained in PMIS. 
This comparison would provide TSA with information to help provide 
reasonable assurance that FSDs and their staffs are complying with the 
standard operating procedures' requirement to call for permission to 
exceed 2 hours' use of the procedures. TSA headquarters officials stated 
that there had not been any instances in which airports were denied 
permission to exceed 2 hours' use of alternative screening procedures. 
However, without records of the calls, we were not able to verify that 
permission was granted for the occurrences that exceeded 2 hours. 38 



TSA Has Taken Action 
to Reduce the Need to 
Use Alternative 
Screening 

Procedures, but Has 
Not Implemented 
Performance 
Measures or Targets 



TSA has taken steps to reduce airports' need to rely on the use of 
alternative baggage screening procedures and is working to minimize the 
need to use these procedures. According to our review of PMIS data, the 
use of alternative screening procedures between October 2004 and 
September 2005 — measured in terms of the total hours these procedures 
were employed — initially increased and then declined. TSA attributed the 
reported overall decline in the usage of alternative screening procedures 



38 TSA classified the number of occurrences that were more than 2 hours as sensitive 
security information. 



TSA Has Taken Steps to 
Reduce the Need to Use 
Alternative Screening 
Procedures at Airports 
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in part to improved coordination among FSDs, airlines, and local 
organizations. According to TSA officials, this coordination helps FSDs 
and their staffs anticipate surges in passenger traffic so that they can 
adequately staff screening stations. In our structured interviews with FSDs 
and their staffs responsible for 29 airports, several FSDs also cited the 
importance of coordination with local organizations and how this reduced 
their need to use alternative screening procedures. For example, the FSD 
for one airport said that he coordinated with local summer camps to have 
campers' baggage screened the day before their flights to reduce the 
amount of baggage that has to be screened when campers arrive at the 
airport. The FSD for another airport communicated with cruise ship 
management about the scheduling of cruises in order to anticipate any 
surges in passenger traffic that may have created the need to use 
alternative screening procedures. 

TSA officials have also taken action to reduce airports' need to use 
alternative screening procedures through the use of "optimization team" 
visits to airports. These visits are conducted at the request of TSA senior 
leadership or an FSD with the goal to observe screening operations and 
maximize efficiencies by applying practices learned at other airports. 
According to TSA officials, recurring use of alternative screening 
procedures triggered some of the optimization team visits TSA has 
conducted, and the optimization team visits may have led to a reduction in 
the number of occasions in which these procedures needed to be used. 39 
For example, on May 18-19, 2005, an optimization team visited one airport 
and recommended procuring one ETD machine and changing the location 
of another to reduce the airport's need to use alternative screening 
procedures. On July 6-7, 2005, another optimization team visited another 
airport. The team suggested reconfiguring EDS machines at the airport's 
screening stations, which resulted in an increase in baggage throughput 
from 120 to 150 bags per hour. At both of these airports, alternative 
screening procedures were used more frequently prior to the optimization 
team visit than they were after the visit. 

TSA officials also stated that as additional equipment is deployed and 
enhanced to enable TSA to increase checked baggage screening 
throughputs — bags screened per hour — TSA will be in a better position to 



39 The optimization visits are also used to improve the design of passenger and baggage 
checkpoints, validate the TSO staffing model at the airport, evaluate staffing and 
scheduling practices, and determine compliance with the standard operating procedures. 
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reduce the need for use of alternative screening procedures. We reported 
in March 2005 that as of June 2004, TSA had deployed 1,228 EDS 
machines. 40 Between June 2004 and June 2006, TSA had deployed 
399 additional EDS machines in both in-line (integrated into the airport 
baggage system) and stand-alone (in airport lobbies or baggage makeup 
areas) configurations. EDS machines in an in-line configuration are able to 
screen up to 500 bags per hour, as compared to EDS machines in a stand- 
alone configuration that screen between 80 and 180 bags per hour. The 
superior efficiency of screening with in-line EDS compared to screening 
with stand-alone EDS may have been a factor in reducing the need to use 
alternative screening procedures at airports where in-line systems were 
installed. TSA reported that, as of June 2006, 25 airports had operational 
in-line EDS systems and an additional 24 airports had in-line systems 
under construction. Although in-line EDS systems can create 
improvements in operational efficiencies of an airport's checked baggage 
screening system, baggage volumes that exceed the system's capacity and 
equipment breakdowns still sometimes occur, necessitating the use of 
alternative screening procedures. For example, some of the airports that 
have installed airportwide in-line systems reported using alternative 
screening procedures because of equipment failures and high passenger 
and baggage volumes after their systems were operational. 41 Since stand- 
alone EDS machines screen between 80 and 180 bags per hour compared 
to ETD machines, which allow for screening of 36 bags per hour, 
additional stand-alone EDS machines also may have helped the airports 
where they were installed to screen baggage with standard screening 
procedures rather than alternative screening procedures. Additionally, in 
May 2005, TSA certified software and hardware upgrades for 519 out of 
1,322 EDS machines, which are used in both in-line and stand-alone 
configurations. These upgrades are being tested in a pilot program. TSA 
officials anticipate that the upgrades could lead to increased baggage 
throughput for the machines, which could further reduce need to use 
alternative screening procedures. 

Installation of in-line EDS systems at airports that currently use stand- 
alone EDS and ETD for primary screening has further potential to reduce 



4U GAO-05-365. 

41 Each of the airports' in-line systems became operational on a different date. The number 
of airports that reported using alternative screening procedures due to equipment failures 
and high passenger and baggage volumes after their systems were operational has been 
determined to be sensitive security information. 
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the need for alternative screening procedures to be used at these airports. 
In March 2005, we reported that TSA had estimated that in-line checked 
baggage systems would reduce by 78 percent the number of baggage TSOs 
and supervisors required to screen checked baggage at nine airports that 
had signed agreements to develop the systems. 42 Under the congressionally 
imposed 45,000 TSO full-time-equivalent limit, when staff requirements are 
reduced at one airport through increased efficiencies, full-time equivalent 
positions will become available to address TSO shortages at other 
airports. 43 TSA's February 2006 checked baggage strategic planning 
framework included a prioritization of which additional airports should 
receive funding for in-line systems and a description of how stand-alone 
EDS machines from those airports should be redistributed to other 
airports. 44 In the framework, TSA also reported that many of the initial in- 
line systems had produced a level of TSO labor savings insufficient to 
offset up-front capital costs of constructing the systems. According to 
TSA, the facility and baggage handling system modifications have been 
higher than expected at the nine airports that have signed agreements to 
fund the systems. TSA stated that the keys to reducing future costs are 
establishing guidelines outlining best practices and a set of efficient design 
choices and using newer EDS technology that best matches each optimally 
scaled design solution. In February 2006, TSA reported that recent 
improvements in the design of the in-line EDS checked baggage screening 
systems and the EDS screening technology now offer the opportunity for 
higher-performance and lower-cost screening systems. The final strategic 
plan will include the results of TSA's cost sharing study that it is currently 
conducting in consultation with airport operators, airlines, and other key 
stakeholders to identify ways to fund in-line EDS systems. 45 After in-line 



"^GAO-05-365. 

43 Section 4023 of the Intelligence Reform and Terrorism Prevention Act of 2004 requires 
TSA to develop and submit to the appropriate congressional committees, standards for 
determining aviation security staffing at commercial airports no later than 90 days after 
December 17, 2004, the date of the act's enactment, and GAO to conduct an analysis of 
these standards. These standards were submitted to Congress on June 22, 2005, and GAO is 
currently reviewing these standards. 

44 The details of the strategic planning framework for the checked baggage screening 
program constitute sensitive security information. 

45 Section 4019(d) of the Intelligence Reform and Terrorism Prevention Act of 2004 requires 
TSA to complete a cost-sharing study in collaboration with industry stakeholders to review 
the benefits and cost of in-line checked baggage screening systems, innovative financing 
approaches, formulas for cost sharing between different government entities and the 
private sector, and potential cost-saving approaches. 
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EDS systems are installed and staffing reductions are achieved, 
redistributing the TSO positions to other airports with staffing shortages 
may reduce airport need to use alternative screening procedures. 

Technology developments may also help TSA to reduce the use of 
alternative screening procedures. In March 2005, we reported that TSA 
was working to develop a computer-aided tomography explosives 
detection system that is smaller and lighter than systems currently 
deployed in airport lobbies and that the new system was intended to 
replace systems currently in use, including larger and heavier EDS 
machines and ETD equipment. 46 The smaller size of the system would 
create opportunities for TSA to transfer screening operations to other 
locations, such as airport check-in counters. The machine would also be 
an option for airports that currently rely on ETD machines since it would 
be cheaper than other certified machines and it would have higher 
baggage throughput than screening using ETD machines, potentially 
reducing the need to use alternative screening procedures at airports 
where it is installed. In March 2005, TSA began to pilot this machine at 
three airports. At one of these pilot airports, the FSD stated that he 
anticipates that the smaller EDS machine will reduce staffing needs, 
reduce workers compensation claims, and ultimately enable the airport to 
incorporate the machines in-line behind the ticket counters. TSA reported 
that the machine achieved throughput rates of up to 80 bags per hour, 
higher than the throughput rate of up to 36 bags per hour for an ETD 
operated by one TSO or up to 72 bags per hour for an ETD operated by 
two TSOs. In September 2005, TSA entered into a $24.8 million contract to 
purchase 72 of these machines that will be installed at 24 airports. 

While TSA data indicate that the use of alternative screening procedures is 
declining, TSA reported in its February 2006 framework that at some 
airports alternative screening procedures will increasingly be used 
because of rising passenger traffic. TSA has projected that the number of 
originating domestic and international passengers will rise by about 
127 million passengers over current levels by 2010. If TSA's current 
estimate of an average of 0.76 checked bags per passenger were to remain 
constant through 2010, TSA would be screening about 96 million more 
bags than it now screens. This could increase airports' need to rely on 
alternative screening procedures in the future in the absence of additional 
or more efficient EDS machines. TSA headquarters officials stated that 



4,J GAO-05-365. 
Page 29 



GAO-06-869 Aviation Security 



while TSA is working to minimize the need to use alternative screening 
procedures, it intends to maintain the procedures as part of its standard 
operating procedures so that FSDs will have options to respond to events 
such as unforeseen equipment failures, surges in passenger traffic, and 
weather-related incidents such as hurricanes. Additionally, some of the 
FSDs that we interviewed stated that they anticipate continuing to need to 
use alternative screening procedures because of screening capacity limits 
and rising passenger volume, and some of these FSDs anticipated 
increasing their use of the procedures as their airport passenger traffic 
rises because of limitations in the physical layout of their airports that 
contribute to overcrowding. 47 



TSA Has Not Established 
Performance Measures or 
Targets Related to the Use 
of Alternative Screening 
Procedures 



Although TSA is working to minimize the need to use alternative screening 
procedures at airports, it has not established performance measures or 
targets related to the use of these procedures. The Government 
Performance and Results Act of 1993 provides, among other things, that 
federal agencies establish program performance measures, including the 
assessment of relevant outputs and outcomes of measures. 48 Performance 
measures are meant to cover key aspects of performance and help 
decision makers to assess program accomplishments and improve 
program performance. A performance target is a desired level of 
performance expressed as a tangible, measurable objective, against which 
actual achievement will be compared. By analyzing the gap between target 
and actual levels of performance, management can target those processes 
that are most in need of improvement, set improvement goals, and identify 
appropriate process improvements or other actions. 



TSA has established four performance measures for the checked baggage 
screening program. Three of these measures make up TSA's checked 
baggage screening performance index. This index measures the overall 
performance of the system through a composite of indicators that are 
derived by combining specific performance measures related to checked 



TSA determined that the exact number of FSDs that anticipate continued or increased use 
of alternative screening procedures is sensitive security information. 

48 According to the Government Performance and Results Act, the Office of Management 
and Budget, and GAO, outcomes assess actual results as compared with the intended 
results that occur from carrying out a program or activity. Outcomes are the results of a 
program or activity. For further information, see GAO, Results-Oriented Government: 
GPRA Has Established a Solid Foundation for Achieving Greater Results, GAO-04-38 
(Washington, D.C.: Mar. 10, 2004). 
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baggage screening. Specifically, this index measures the effectiveness of 
screening systems through machine probability of detection and covert 
testing results, efficiency through a calculation of dollars spent per bag 
screened, and customer complaints at both airports and TSA's national 
call center. TSA considers the final performance measure — compliance 
with the ATSA requirement to screen all checked baggage using explosive 
detection systems (EDS and ETD) — to be obsolete since it reported all 
airports as capable of screening with EDS or ETD in January 2005. The use 
of alternative screening procedures is not included in the index, nor does 
TSA have stand-alone measures or targets for the use of alternative 
screening procedures. 

TSA officials stated that they did not want to implement performance 
measures or targets for alternative screening procedures because they are 
already working to minimize the need to use the procedures at airports. 
However, TSA officials also acknowledged that they will continue to rely 
on alternative screening procedures because of unforeseen circumstances 
such as high baggage volumes or weather-related incidents. By creating a 
performance measure for the use of alternative screening procedures as 
part of the checked baggage screening index or as a stand-alone measure, 
TSA could gauge whether it is making progress toward minimizing the 
need to use these procedures at airports and have more complete 
information on how well the overall checked baggage screening system is 
performing. Furthermore, performance targets for the use of alternative 
screening procedures would provide an indicator of how much risk TSA is 
willing to accept in using these procedures, and TSA's monitoring of this 
indicator would identify when it has exceeded the level of risk that it has 
determined is acceptable. For example, if TSA were to determine the 
percentage of checked baggage that should be screened using alternative 
screening procedures, and if its performance data showed that it was 
currently screening a higher percentage than the target, TSA would be able 
to decide whether to take steps to bring the use of these procedures into 
line with its desired level of use. 

Finally, the extent to which performance measures and targets will assist 
TSA in minimizing the need to use these procedures at airports is 
dependent upon the accuracy and completeness of the reporting of 
alternative screening procedures in PMIS, including the percentage of bags 
screened using the procedures, as previously discussed. 



It has been over 4 years since Congress issued the mandate for TSA to 
screen all checked baggage at commercial airports using explosive 
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detection systems. During this time, TSA has deployed EDS or ETD 
machines at more than 400 commercial airports and reported achieving 
the capability to screen 100 percent of checked baggage using these 
machines. As part of this effort, TSA developed standard and alternative 
checked baggage screening procedures. While TSA acknowledges that 
screening with alternative screening procedures is less effective than 
screening with standard screening procedures, it has also recognized the 
need for continued use of alternative screening procedures because of 
high passenger and baggage volumes resulting from unpredictable and 
unforeseen circumstances, such as equipment breakdowns and unusual 
weather events. Given TSA's plans to continue to use alternative screening 
procedures and the trade-offs in security effectiveness involved in their 
use, it will be important for TSA to test the effectiveness of these 
procedures in an operating environment. One such way for TSA to test the 
security effectiveness of the various alternative screening procedures is 
through the covert testing conducted by the Office of Inspections. While 
we recognize the logistical challenges that the Office of Inspections faces 
in conducting checked baggage covert testing on alternative screening 
procedures, using PMIS data on the use of these procedures — including 
data on the airports that use the procedures the most frequently or for 
extended periods of time — could help the Office of Inspections in 
selecting airports for testing as part of the risk-based approach to covert 
testing that it is currently developing. By not assessing alternative 
screening procedures through national or local covert testing, TSA is 
missing an opportunity to gather information to help determine the 
security effectiveness of alternative screening procedures in an 
operational setting. 

Additionally, TSA headquarters has established internal controls to 
monitor and track the use of alternative screening procedures at airports 
and has taken steps to improve reporting of these procedures in the PMIS 
database. However, without strengthening its controls, such as providing a 
means for measuring the number of bags screened using alternative 
screening procedures and enabling TSA airport staff to report the 
concurrent use of more than one alternative screening procedure, TSA 
lacks reasonable assurance that it has complete and accurate information 
on the use of these procedures. 

Furthermore, TSA has taken steps to reduce the need to use alternative 
screening procedures at airports, but does not expect to eliminate the use 
of these procedures. Increasing air travel and TSA's effort to operate 
within or below the current 45,000 TSO full-time-equivalent limit could add 
to the need for alternative screening procedures, unless more or more 
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efficient EDS machines are deployed. Given TSA's continuing use of the 
procedures, performance measures and targets would provide TSA and 
Congress with objective information to assess TSA's progress in 
minimizing the need to use the procedures at airports, and would help 
inform TSA decision making on whether and when mitigating steps are 
needed to achieve its desired level of use. 



Recommendations for ^° m ^ orm ^SA of the security effectiveness of alternative screening 

procedures in an operational setting, and to help TSA strengthen its 

Executive Action monitoring of the use of alternative screening procedures, we recommend 

that the Secretary of the Department of Homeland Security direct the 
Assistant Secretary, Transportation Security Administration, to take the 
following four actions: 

• Use PMIS data on use of alternative screening procedures in determining 
at which airports to conduct covert testing and when to conduct testing at 
these airports as part of the Office of Inspections' new risk-based 
approach to covert testing. 

• Conduct local covert testing of alternative screening procedures to 
determine whether checked baggage TSOs can detect simulated 
improvised explosives when using these procedures. 

• Strengthen the monitoring and tracking of the use of alternative screening 
procedures to help determine the progress the agency is making in 
minimizing its need to use these procedures. This effort would include 
continuing to address reporting problems in the PMIS database system, 
keeping a record of calls requesting permission to exceed 2 hours' use of 
the procedures, and providing a means for measuring the use of 
alternative screening procedures compared to the use of standard 
procedures, such as counting baggage screened with alternative screening 
procedures. 

• Develop performance measures and performance targets for the use of 
alternative screening procedures in checked baggage screening, perhaps 
as part of the checked baggage screening program performance index, to 
help TSA measure its progress in working toward minimizing the need to 
use alternative screening procedures at airports and to have more 
complete information on the overall performance of the checked baggage 
screening system. 
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Agency Comments 
and Our Evaluation 



We provided a draft of this report to DHS for review and comment. On 
July 25, 2006, we received written comments on the draft report, which are 
reproduced in full in appendix II. DHS concurred with our findings and 
recommendations and stated that the report will help strengthen TSA's 
management of checked baggage screening operations. 

Regarding our recommendation that TSA use Performance Management 
Information System data on the use of alternative screening procedures in 
determining at which airports to conduct covert testing and when to 
conduct testing at these airports, DHS concurred and stated that TSA's 
Office of Inspections will consider PMIS information on alternative 
screening procedures as part of its new risk-based approach to covert 
testing and will develop new checked baggage screening testing protocols. 
Concerning our recommendation that TSA conduct local covert testing of 
alternative screening procedures, DHS concurred and stated that TSA is 
currently modifying its local covert testing program to strengthen the 
program and expects that these modifications will better prepare TSOs to 
detect simulated improvised explosives. We are pleased that TSA is 
making efforts to strengthen its local covert testing program. We continue 
to believe that testing of alternative screening procedures would provide 
TSA with an opportunity to gather information to help identify and 
improve the security effectiveness of alternative screening procedures in 
an operational setting. 

DHS concurred with our recommendation to strengthen the monitoring 
and tracking of the use of alternative screening procedures to help 
determine the progress the agency is making in minimizing its need to use 
the procedures. In response to our recommendation that TSA address 
reporting problems in its PMIS database system, DHS stated that PMIS has 
been enhanced with both functionality and data quality-related processes 
to ensure data reliability. According to TSA, the system alerts the user 
when a data field is filled in with a value that falls outside the operational 
norms for a particular airport. Additionally, according to TSA, PMIS 
training and functionality reviews occur on a regular basis and user 
manuals and best practices are updated consistently. While these efforts 
should help improve the data reliability of PMIS, they will not fully address 
the reporting problems highlighted in our report. Specifically, the steps 
TSA has taken do not address inaccurate counts of baggage screened 
resulting from TSA's method for estimating the number of bags screened 
with ETD using standard screening procedures or inaccurate reporting of 
occurrences when a particular alternative screening procedure is used. 
Without addressing these reporting problems, TSA will continue to lack 
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reasonable assurance that it has complete and accurate information on the 
use of these procedures. 

In response to our recommendation on keeping a record of calls 
requesting permission to exceed 2 hours use of the procedures, DHS 
stated that because the amount of time that alternative screening 
procedures are used is recorded in PMIS, there is no further 
documentation required for exceeding the 2 hour threshold. However, 
while documentation is entered into PMIS on the amount of time the 
procedures are used, recording the length of time that the procedures are 
used does not allow TSA to verify that FSDs are actually requesting 
permission to use the procedures for more than 2 hours as required. In 
response to our recommendation on providing a means for measuring the 
use of alternative screening procedures compared to the use of standard 
procedures, such as counting baggage screened with alternative screening 
procedures, DHS stated that TSA will evaluate the necessity of requiring 
the recording of the number of bags screened by alternative screening 
procedures, and if it finds it to be a useful metric, it will require FSDs to 
include the number of bags in their report in PMIS. We are encouraged 
that TSA will undertake this evaluation, as we believe that it will allow 
TSA to have more assurance that it has complete and accurate information 
on the use of these procedures. 

In response to our recommendation on developing performance measures 
and targets for the use of alternative screening procedures, DHS 
concurred and stated that TSA is currently meeting the intent of this 
recommendation by monitoring and tracking the use of alternative 
screening procedures through PMIS. DHS stated that using this system has 
assisted TSA in identifying areas for improvement nationwide and 
addressing local issues to minimize the need for alternative screening 
procedures. DHS also stated that TSA intends to continue monitoring and 
tracking the use of alternative screening procedures and to implement the 
recommendations in this report for refining the data and evaluating the 
need to make adjustments based on the current performance level. While 
we support TSA's efforts to ensure the use of alternative screening 
procedures is accurately reported in PMIS, given the security effectiveness 
trade-offs associated with alternative screening procedures, we do not 
believe that tracking the use of the procedures with PMIS is sufficient to 
provide congressional and other decision makers with an indication of the 
progress the agency expects to make in minimizing the need to use the 
procedures at airports. Performance measures and targets would provide 
this information and would help to reinforce accountability and to ensure 
that managers focus on the results they are striving to achieve regarding 
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minimizing the use of alternative screening procedures in their day-to-day 
activities. 



We will send copies of the report to the Secretary of the Department of 
Homeland Security; the Assistant Secretary, TSA; and interested 
congressional committees as appropriate. We will also make copies 
available to others on request. In addition, the report will be available at 
no charge on GAO's Web site at http://www.gao.gov. 

If you or your staff have any questions about this report, please contact me 
at (202) 512-3404 or berrickc@gao.gov. Contact points for our Offices of 
Congressional Relations and Public Affairs may be found on the last page 
of this report. GAO staff that made major contributions to this report are 
listed in appendix III. 

Sincerely yours, 




Cathleen A. Berrick 
Director, Homeland Security and 
Justice Issues 
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Appendix I: Objectives, Scope, and 
Methodology 



To assess the Transportation Security Administration's (TSA) efforts to 
screen all checked baggage using explosive detection systems (EDS) and 
explosive trace detection machines (ETD), we addressed the following 
questions: (1) How did TSA prioritize the use of standard and alternative 
checked baggage screening procedures and what security effectiveness 
trade-offs and operational efficiencies has TSA identified in using these 
procedures to screen checked baggage for explosives? (2) To what extent 
has TSA used standard and alternative screening procedures to screen 
checked baggage for explosives and how does TSA ensure that standard 
screening procedures are used whenever possible? (3) What steps has TSA 
taken to reduce airports' need to use alternative screening procedures and 
to establish performance measures and targets for the use of the 
procedures? 

To assess how TSA prioritized the use of checked baggage screening 
procedures and to assess the trade-offs in security effectiveness and 
operational efficiencies associated with various baggage screening 
procedures, we analyzed TSA's standard operating procedures for using 
these procedures. We also obtained and analyzed relevant legislation and 
conducted a literature search to obtain information on screening 
procedures, technologies, and related aviation trends. This search 
identified various TSA reports, Department of Homeland Security 
Inspector General reports, and aviation industry reports. We also reviewed 
studies from the TSA Transportation Security Laboratory regarding 
checked baggage screening. We interviewed officials from various TSA 
offices, including the Chief Technologist's Office, Aviation Programs, the 
Transportation Security Operations Center, the Transportation Security 
Laboratory, Chief Operating Officer's Office, and Office of Planning to 
learn about checked baggage screening procedures and how they were 
given relative priority. We also interviewed officials from air carriers, 
explosive detection systems equipment manufacturers, and an airport 
industry association to obtain information regarding TSA's checked 
baggage screening procedures. We assessed the results from 
unannounced, undercover covert testing of checked baggage screening 
operations conducted by TSA's Office of Inspections and questioned TSA 
officials about the procedures used to ensure the reliability of the covert 
test data. On the basis of their answers, we believe that the covert test data 
are sufficiently reliable for the purposes of our review. We also reviewed 
results of unannounced, undercover covert testing of checked baggage 
screening operations conducted at airports by Federal Security Directors 
(FSD) and their staffs and collected as part of TSA's Screener Training 
Exercises and Assessments program. After reviewing documentation from 
TSA, we found the data from the Screener Training Exercises and 
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Appendix I: Objectives, Scope, and 
Methodology 



Assessments program were sufficiently reliable for the purposes of our 
review. 

To assess the extent to which TSA has used standard and alternative 
screening procedures to screen checked baggage and how TSA ensures 
that standard checked baggage screening procedures are used whenever 
possible, we reviewed and analyzed TSA's Performance Management 
Information System (PMIS) database, which contains information on 
baggage screening operations and the use of alternative screening 
procedures. We found several issues with these data, including, in some 
cases, multiple occurrences of the use of alternative screening procedures 
recorded as one occurrence and also, in some cases, more than one 
procedure being used during an occurrence but the occurrence was 
entered into the database as only one procedure because of the 
constraints of the database. When we interviewed TSA officials about 
these data reliability issues, officials acknowledged that airports may have 
inaccurately reported some occurrences of the use of alternative screening 
procedures. However, the officials stated that they were working to 
correct the reporting problems and consider the data generally reliable. On 
the basis of these discussions and our review of the database, we found 
the data to be sufficiently reliable for the purposes of this report, since the 
data provide overall trends in the use of the procedures. To determine 
what controls are in place to track and report the use of baggage screening 
procedures, we analyzed the PMIS database and the PMIS user guide. We 
also analyzed TSA's operating procedures for checked baggage and policy 
guidance and compared TSA's procedures for ensuring that airports 
correctly report the use of alternative screening procedures to the 
Comptroller General's Standards for Internal Controls in the Federal 
Government. We also interviewed officials from TSA's Office of Planning, 
Chief Operating Office, Transportation Security Operations Center, and 
Inspections offices concerning checked baggage screening procedures. 

To assess the steps TSA has taken to reduce airports' need to use 
alternative screening procedures and the measures and targets TSA has set 
for alternative screening procedures, we analyzed TSA's PMIS data and its 
standard operating procedures for checked baggage screening and TSA's 
inventory of explosive detection systems. We found discrepancies in the 
inventory data of explosive detection systems and worked with TSA to 
resolve the discrepancies. TSA also completed a reconciliation of the 
inventory database with data collected manually by TSA officials. We also 
analyzed documentation from the TSA Transportation Security Operations 
Center and interviewed TSA officials from the Chief Operating Officer's 
Office, Office of Planning, the Office of Assistant Secretary, and Chief 
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Appendix I: Objectives, Scope, and 
Methodology 



Technology Office. Additionally, we examined TSA's checked baggage 
performance measures and targets in the context of the Government 
Performance and Results Act of 1993 (GPRA) requirements. 

In addressing these objectives, we conducted site visits at nine airports — 
three category X, one category I, four category II, and one category IV 
airport. We chose these airports based on one or more of the following 
factors: use of alternative screening procedures at the airport as reported 
in PMIS, testing of screening equipment at the airport, proximity to 
another airport being visited by GAO, and size of airport. The results from 
our airport visits provided examples of checked baggage screening 
operations and issues but cannot be generalized beyond the airports 
visited because we did not use statistical sampling in selecting the 
airports. We also conducted structured interviews with FSDs and their 
staffs who were responsible for 29 randomly selected airports. One FSD 
we interviewed was responsible for two airports in our sample. We 
conducted all but one of these interviews over the telephone. Using 
information from PMIS, we selected airports that had reported using 
alternative screening procedures and airports that had not reported using 
alternative screening procedures between October 18, 2004, and December 
21, 2004. 1 Although the interviews were conducted with FSDs and then- 
staffs at random samples of airports, the samples are too small to 
generalize the interview results with a high degree of statistical confidence 
to all airports nationwide. The results from these interviews do provide 
information about checked baggage screening operations at the airports 
for which the FSDs and their staffs are responsible. 

We conducted our work from September 2004 through July 2006 in 
accordance with generally accepted government auditing standards. 



X TSA has determined that the exact number of airports we selected is sensitive security 
information. 
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Appendix II: Comments from the Department 
of Homeland Security 



U.S. Department of Homeland Security 

Washington, DC 20528 



iwwh Homeland 
'WW J Security 

July 25, 2006 



Ms. Cathleen A. Berrick 

Director, Homeland Security and Justice Issues 

U.S. Government Accountability Office 

441 G Street, NW 

Washington, DC 20548 

Dear Ms. Berrick: 

Thank you for the opportunity to comment on the Draft Report, "Aviation Security: TSA 
Oversight of Checked Baggage Screening Procedures Could Be Strengthened," GAO-06- 
869, the public version of GAO-06-291SU, which was issued on February 28, 2006. The 
recommendations and findings of this report should help strengthen TSA's management 
of checked baggage screening operations. 

We are pleased to report that TSA has already undertaken initiatives to improve checked 
baggage screening operations. For example, TSA has refined its definitions and 
requirements for using alternative screening procedures (ASP). As such, the amount of 
total checked baggage screening time using ASP is very low. In addition, TSA has 
deployed sufficient screening equipment to meet its requirement to screen all checked 
baggage using explosives detection equipment, refined staffing levels, and optimized 
screening procedures. 

Covert checked baggage tests conducted by the TSA Office of Inspection (OI), formerly 
the Office of Internal Affairs and Program Review (OIAPR), are designed to identify 
vulnerabilities in the checked baggage screening system. OI covert checked baggage 
tests were originally designed to evaluate the effectiveness of TSA's newly hired and 
trained screener workforce and to assess the adequacy of security systems and controls at 
all airports nationwide. OI committed to complete covert checkpoint and checked 
baggage testing of all TSA and Screening Partnership Program airports in a 3-year 
timeframe. During the period of this review, OI did not focus on ASP. As part of TSA's 
new risked-based approach to covert testing, OI will consider new checked baggage 
screening testing protocols. 

The following represents the Department of Homeland Security (DHS) and TSA's 
responses to the recommendations contained in the draft report. 



www.dhs.gov 
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Appendix II: Comments from the Department of Homeland 
Security 



Recommendation 1: Use Performance Management Information System (PMIS) 
data on use of alternative screening procedures in determining at which airports to 
conduct covert testing and when to conduct testing at these airports as part of the 
Office of Inspection's new risk-based approach to covert testing. 

Concur: The Office of Inspection will consider PMIS information on alternative 
screening procedures as part of TSA's new risk-based approach to covert testing. 
Accordingly, we will develop new checked baggage screening testing protocols that will 
assist TSA in strengthening aviation security. 

Recommendation 2: Conduct local covert testing of alternative screening 
procedures to determine whether checked baggage Transportation Security Officers 
(TSO) can detect simulated improvised explosives when using these procedures. 

Concur. TSA is currently making modifications to strengthen the local covert testing 
programs. TSA expects these modifications will better prepare TSOs to detect simulated 
improvised explosives. 

Recommendation 3: Strengthen the monitoring and tracking of the use of 
alternative screening procedures to help determine the progress the agency is 
making in minimizing its need to use these procedures. This effort would include 
continuing to address reporting problems in the PMIS database system, keeping a 
record of calls requesting permission to exceed 2 hours' use of the procedures, and 
providing a means for measuring the use of alternative screening procedures 
compared to standard procedures, such as counting baggage screened with 
alternative screening procedures. 

Concur: The Performance Management Information System (PMIS) is enhanced with 
both functionality and data quality related processes to ensure data reliability. The 
system alerts the user when a data field is filled in with a value that falls outside of the 
operational norms for a particular airport. Additionally, PMIS training and functionality 
reviews occur on regular bases and user manuals and best practices are updated 
consistently. 

As stated in the report, TSA requires all Federal Security Directors (FSD) to report any 
instance and type of alternative screening procedures. Currently, all instances, types, and 
length of time using alternative screening procedures are documented in PMIS. 
If the procedures are needed for more than 2 hours, the Assistant Administrator of 
Security Operations is notified and speaks to the FSD requesting permission to extend the 
procedures for more than 2 hours. Since the amount of time alternative screening 
procedures are used is recorded in PMIS, there is no further documentation required for 
exceeding the 2-hour threshold. TSA will evaluate the necessity of requiring the 
recording of the number of bags screened by ASP and if found to be a useful metric, will 
require FSD's to include the number of bags in their report in PMIS. 
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Appendix II: Comments from the Department of Homeland 
Security 



Recommendation 4: Develop performance measures and performance targets for 
the use of alternative screening procedures in checked baggage screening, perhaps 
as part of the checked baggage screening program performance index, to help TSA 
measure its progress in working toward minimizing the need to use alternative 
screening procedures at airports and to have more complete information on the 
overall performance of the checked baggage screening system. 

Concur: TSA is currently meeting the intent of this recommendation by monitoring and 
tracking the use of alternative screening procedures through PMIS. Using this system has 
assisted TSA in identifying areas of improvement nationwide and addressing local issues 
to minimize the need for alternative screening procedures. Consequently, TSA is 
spending a very low percent of its total checked baggage screening time using alternative 
screening procedures. 

TSA intends to continue monitoring and tracking the use of alternative screening 
procedures and to implement the recommendation in this report for refining the data we 
capture to assist us in evaluating the need to make adjustments based on our current 
performance level. 

Thank you for the opportunity to provide comments on your draft report. The 
Department and TSA appreciate the amount of time and work invested in this type of 
study. 



Sincerely, 




Steven J. Pecinovsky 

Director Departmental GAO/OIG Liaison Office 
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